The use of advanced vision systems (also termed “machine vision”) and their underlying software is increasingly employed in a variety of manufacturing and quality control processes. Machine vision enables quicker, more accurate and repeatable results to be obtained in the production of both mass-produced and custom products. Typical machine vision systems include one or more cameras (typically having solid-state charge couple device (CCD) or CMOS-based imaging elements) directed at an area of interest, a frame grabber/image processing elements that capture and transmit CCD images, one or more computers or onboard processing devices, and a user interface for running the machine vision software application and manipulating the captured images, and appropriate illumination on the area of interest.
The use of multiple cameras (also termed “sensors”), is desirable in many vision system applications. Such a multi-camera machine vision system arrangement can allow for a part or object on a moving line, to be simultaneously or sequentially imaged from several vantage points, increasing the reliability of the inspection process by taking into account features in certain views that may be occluded in other views. Thus, the use of multiple image processing devices, often one per camera can allow for faster line speeds, as multiple processing devices may work simultaneously to inspect a single part or group of parts. The processing devices may be physically adjacent or they may be widely separated and connected via a network or other communication channel. This is known as a distributed system or application because the processing capability is physically distributed among multiple devices. Greater line speeds are possible using additional processing devices, such that each device or group of devices inspects one of a plurality of adjacent objects in each inspection cycle (typically governed by a clock and trigger signal). For example, in a given inspection cycle a first camera or group of cameras images a first part (a discrete object or a region of an object or surface), a second camera/camera group simultaneously images the next adjacent part on the line and a third camera/camera group simultaneously images a third adjacent part on the line. Multiple processing devices, typically one per camera or one per group of cameras, work simultaneously to inspect all three parts at once. This is known as a multi-phase system where, in this example, three inspection phases are carried out simultaneously within a single inspection cycle.
Generally, in a distributed machine vision application, multiple processing subsystems acquire and process respective multiple images pertaining to a single part or inspection cycle and communicate these results over a network or other communication channel. In order to generate a valid overall result for a given inspection cycle, the results generated by all processors in the distributed system are combined in such a way that ensures that all result values pertain to the same inspection cycle. Since each inspection result can have an unpredictable processing time, it is sometimes difficult to determine if a given result pertains to a specific inspection cycle or another inspection cycle. The system as a whole must have a means to automatically detect and recover from transient problems such as missed or extra trigger signals, temporary power interruption to one or more subsystems, and unreliable communication channels. In applications that require substantial processing power it may be necessary to distribute a single inspection task among multiple processors, which renders synchronizing the results even more challenging.
By way of background, many manufacturers of machine vision technology offer multi-camera systems in the literal sense—multiple cameras, all feeding a single processor box, which is typically defined by a PC, or a standalone processing unit. The central processing unit (CPU) in such a multi-camera system controls when the cameras are triggered to acquire an image, and stores the acquired images in memory, whereby the application can associate the appropriate images to the application.
Many distributed multi-camera applications employ a programmable logic controller (PLC) to synchronize and combine results from discrete vision systems in the overall arrangement. In the absence of any synchronization capability within the vision systems themselves, this is the standard solution to the problem of synchronization.
Certain vision systems, such as the In-Sight® product available from Cognex Corporation of Natick, Mass. include a WriteResult and ReadResult command within their vision system applications. In-Sight is a self-contained camera assembly with an onboard processing device, capable of communicating with a network via appropriate networking protocols, such as Ethernet. The WriteResult and ReadResult commands allow discrete, interconnected vision systems to share data over a network. In this manner, one vision system can act as a master processor, and one or more others can act as slave processors. If one of the vision systems is a master camera, it can be used to trigger slave camera(s) via Ethernet in a way that ensures that the results received from the slave camera(s) are identified as part of the current inspection cycle. In typical arrangements, the combined vision system results for the inspection cycle are commonly communicated from the master camera to a PLC, where they are analyzed and a final result or results is/are determined.
As described above, in order to apply more processing power, existing vision system applications have been solved using duplicate systems triggered in sequential phases. This can be termed a “multi-phase” approach herein. The simplest example uses two cameras and two processors, with each system inspecting every other part. More complex applications can use multiple banks of cameras, for example three banks of cameras and processors each inspecting every third part, with any number of cameras in each bank. It is possible to synchronize and serialize the results externally in a PLC or, with difficulty, using networking data functions within the vision system, such as WriteResult and ReadResult.
Nevertheless, it proves very challenging and time-consuming to provide a reliable PLC application that correctly synchronizes results at high speed and can recover from many types of transient problems that may affect synchronization of the results of a given inspection cycle. Additionally, placing the burden of synchronization on the PLC adds cost and complexity to the application, and makes it more difficult to maintain due to the “vision logic” in the PLC program that typically also controls the process equipment.
One approach to ensuring synchronization is to provide a central processor for all machine vision tasks interconnected to multiple dumb cameras (i.e. cameras that simply acquire images). However, this approach suffers from a lack of scalability. As more cameras are added, additional burden is placed on the processor, which is contemporaneously receiving a plurality of images for machine-vision processing. In some cases it is possible to add co-processors to increase the machine vision processing capability, but it offers very limited scalability compared to a distributed system on a network and it is difficult to use in a multi-phase approach.
Available networked vision system functions, such as WriteResult and ReadResult, tend to provide reliable results only when a master system triggers the slave system via Ethernet/network links and protocols. That is, such functions do not synchronize results when using external, hard-wired trigger signals. The use of external triggers is common where encoders, presence sensors (e.g. photodetectors), and the like are employed to track the motion and location of parts along a moving line and/or regions of interest on a surface, such as a moving web. Each system is triggered when predetermined motion has occurred. One particular concern occurs when a sensor misses a trigger (due, for example to an unreliable photodetector), then the result for that system may not occur and the overall system will thereafter lose synchronization, as the next trigger for that system may be assumed to produce the result for the previous cycle. This limitation severely restricts the speed at which such a master-slave system can operate, and renders it unusable for many applications. Also, these system functions make the application difficult to maintain, because each slave job must receive a WriteResult function that exactly matches the sequence of results expected by the master, and if the sequence is incorrect, there is no reliable way to detect the this error. Moreover, such network-based system functions limit scalability because they are impractical to maintain across a multiplicity of interconnected subsystems. More generally, networked approaches may be too slow for use in high-speed multi-phase, multi-camera arrangements, because prior applications of those approaches were more suited to triggers occurring in the millisecond range, rather than the microsecond time base desired for high-speed operation.
With a multi-phase system using multiple banks of processors each inspecting every Nth part, synchronizing results in a PLC or directly between processors is extremely challenging and often impractical. Overcoming these limitations has, to date, consisted of employing a more-powerful processor if available and cost-effective, accepting the fact that performance will be suboptimal, or otherwise avoiding use of a multi-phase system arrangement.
It is, therefore, desirable to provide a system and method to synchronize inspections in a given cycle, and to combine results from multiple distributed processors into a single processor, in a reliable way. This system and method should allow recovery from most types of intermittent signaling problems and other temporary problems (missing triggers, lost parts, etc.). The arrangement should desirably tolerate unpredictable network delays, and be capable of high speed operation with a variety of triggering methods including hard-wired, external triggering. The system and method should also allow the ability to scale applications to even higher speeds through multi-phase synchronization, in a way that is transparent to the user, and that renders the external communication straightforward.